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ABSTRACT 

The  performance  of  military  systems  (platforms,  sensors  and  weapons)  for  real  world 
operations  is  affected  by  both  natural  environment  conditions  and  human  performance.  As 
shown  in  an  earlier  presentation  (MORSS  71)  human  performance  (decision  making)  is  linked 
directly  to  the  scenario  environment  conditions,  present  and  past.  As  the  Navy  moves  to 
increased  automation  on  board  ships,  the  software  developed  must  incorporate  the  interaction  of 
human  performance  (decision  making)  and  the  scenario  environment  conditions.  The  Systems 
Engineering  Concept  Model  (SECM)  is  being  used  in  several  Navy  programs  to  capture  and 
analyze  all  relevant  entities,  their  attributes  and  their  interactions  in  a  defined  military  scenario, 
first  from  the  real  world  view  and  then  from  the  software  perspective.  The  process  can  be  used  to 
capture  the  functional  requirements  that  insure  the  relevant  interactions  between  the  decision 
making  and  the  scenario  environment  conditions  are  identified. 

INTRODUCTION 

United  States  Navy  ships  are  facing  an  increasing  variety  of  threats,  such  as  high-speed, 
low-flying  anti-ship  cruise  missiles  (ASCM)  that  are  beginning  to  appear  in  the  inventories  of 
many  hostile  forces.  The  Navy  Ship  Self  Defense  Program  is  focused  on  developing  the  ability 
of  a  single  ship  to  defend  itself  against  an  attack  involving  a  single  threat  or  a  stream  of  such 
threats.  To  support  the  Ship  Self  Defense  program,  the  Navy  is  fielding  a  suite  of  highly 
automated  and  integrated  military  systems  to  aircraft  carriers  and  the  Whidbey  Island,  Wasp,  and 
San  Antonio  classes  of  amphibious  assault  ships.  During  the  development  of  these  ships,  it  is  not 
practical  to  test  their  integrated  self  defense  capabilities  against  live  threats,  so  the  Navy  is 
increasingly  using  Modeling  and  Simulation  (M&S)  to  supplement  live  exercises  to  test  and 
evaluate  Ship  Self  Defense.  (Reading  and  Pobat  2000.) 
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In  the  operational  world,  the  natural  environment  plays  a  significant  role  in  determining 
the  performance  of  the  threat  and  ship  defensive  systems  and  how  the  warfighters  utilize  them. 
The  effect  that  environmental  factors  have  on  the  performance  of  the  individual  military  systems 
is  well  understood  from  land  and  sea-based  tests.  For  sensors  such  as  radars,  these  environmental 
factors  include  refraction,  attenuation,  and  clutter.  (Skolnick  1990.)  Ship  Self  Defense,  however, 
involves  the  performance  of  the  complete  ship  system  (the  ship  itself,  the  integrated  military 
systems,  and  the  warfighters)  when  engaged  in  a  self-defense  combat  situation.  The  role  that 
environmental  factors  play  in  Ship  Self  Defense  is  not  as  well  understood  and  is  complicated 
because  the  action  of  complete  ship  system  dining  a  combat  situation  can  alter  the  environmental 
factors  involved. 

M&S  offers  the  option  of  exploring  how  a  wide  range  of  environmental  conditions  can 
affect  ship  system  performance  in  a  self-defense  combat  situation  and  vice-versa  provided  that 
the  simulation  used  appropriately  represents  the  operational  world.  In  order  to  achieve  this  goal, 
the  simulation  must  include  appropriate  models  as  well  as  their  attributes,  behaviors,  and 
interactions  for  the  various  military  systems,  the  ship  itself,  the  warfighters,  and  threats. 

Many  studies  have  been  conducted  on  representing  the  impact  of  environmental  effects 
on  military  systems  in  M&S  applications  (e.g.,  Chadboume  et  al.  1998  and  Douglas  et  al.  2001). 
However,  there  has  been  only  limited  research  into  the  role  that  environmental  factors  play  in  the 
human  decision  making  processes  related  to  the  use  of  military  systems.  One  cognitive  model 
involving  terrain  is  described  in  Juarez-Espinosa  and  Gonzalez  (2004). 1  In  this  paper,  we  address 
the  general  issues  of  what  role  the  environment  can  play  in  human  decision  making  process.  We 
then  address  the  issues  from  a  military  operations  standpoint  to  address  how  the  environment 
needs  to  be  represented  in  the  M&S  tools  being  used  to  develop  and  test  the  performance  for 
Ship  Self  Defense. 

In  this  paper,  we  describe  our  approach  and  assumptions  made  on  how  one  can 
incorporate  human  performance  into  simulations  of  the  Ship  Self  Defense  concept.  Next,  we  pre¬ 
sent  a  “real  world  view”  of  the  decision  making  processes  involved  in  ship  self  defense.  Then, 
we  present  a  conceptual  representation  of  the  human  decision  making  process  from  both  a 
general  perspective  and  how  it  is  represented  in  the  military  domain.  Next,  we  discuss  how  these 
processes  can  be  incorporated  in  an  existing  simulation  system  that  has  been  developed  to 
support  the  Ship  Self  Defense  program.  Finally,  we  present  a  summary  of  these  efforts  and 
where  we  are  proposing  to  go  in  future  work. 

APPROACH  AND  STATEMENT  OF  ASSUMPTIONS 

The  Navy  has  mandated  that  that  specific  ship  classes  must  have  self  defense  capability 
against  a  raid  of  incoming  ASCM  threats  and  that  this  capability  will  be  assessed  using  the 
Probability  of  Raid  Annihilation  (PRA)  as  the  primary  Measure  of  Effectiveness  (MOE).2  The 
PRA  Assessment  Process  Standards  and  Architecture  (PS&A)  has  been  developed  to  provide  an 
approved  roadmap  for  each  ship  class’s  managerial  and  technical  teams  to  use  in  assessing  the 
PRA  MOE  for  that  ship  class.  As  part  of  the  risk  mitigation  strategy,  the  PS&A  is  designed  to 
meet  Operations/Operational  Evaluation  (OPEVAL)  requirements  in  a  consistent  and  adequate 
manner  and  to  reduce  costs  by  using  standard  practices  and  tools  as  well  as  by  building  on 
previous  ship  class  work.3  For  more  details  on  the  PRA  Assessment  PS&A  see  Blake  et  al 
(2003). 
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The  PRA  MOE  is  difficult  to  assess  by  the  traditional  sea-based  and  land-based  testing. 
Cost  is  always  a  limiting  factor  but  a  more  important  consideration  is  the  requirement  to  test  the 
complete  ship  system  along  a  timeline,  not  just  the  individual  components  of  the  ship.  That  is, 
the  PRA  MOE  must  be  assessed  using  complete  scenarios  with  all  relevant  military  components, 
ship  and  threat,  behaving  and  interacting  along  a  timeline  as  they  would  during  actual  ship  self 
defense  situations.  Furthermore,  the  scenarios  must  include  a  reasonable  range  of  environmental 
conditions  that  would  be  appropriate  during  such  ship  self  defense  situations.  For  these  and  other 
reasons,  the  PRA  Assessment  Process  includes  interoperable  simulations,  for  example,  the  PRA 
Assessment  Simulation  Testbed.  The  sea-based  and  land-based  testing  trials  are  incorporated  as 
part  of  the  robust  validation  process.  For  more  information  on  the  ship  self  defense  PRA,  see 
Grigsby  and  Blake  (2001)  and  Reading  and  Sawyer  (2003). 

To  date  the  interoperable  simulations  included  in  the  PS&A  have  not  included  any 
representations  of  the  warfighters.  While  the  new  ship  self  defense  systems  are  highly 
automated,  some  components  still  require  an  EW  operator,  for  example,  the  SLQ-32A  during 
combat  situations.  The  key  human  performance  factor  provided  by  the  EW  operator  is  decision¬ 
making.  The  addition  of  a  human  performance  (decision-making)  model  in  the  Ship  Self  Defense 
simulations  requires  that  the  environmental  representation  be  provided  for  that  model  as  well  as 
for  the  other  ship  and  threat  system  representations.  The  approach  used  in  this  paper  to  establish 
the  human  performance  (decision-making)  and  natural  environment  consistency  across  Ship  Self 
Defense  interoperable  simulations  will  be  an  extension  of  that  developed  for  the  PS&A  to  handle 
military  systems  and  natural  environment. 

The  PS&A  is  based  on  common  systems  engineering  techniques  and  tools.  The  first  steps 
include  defining  the  problem  objectives  and  scope  and  then  establishing  the  requirements  and 
conceptual  views  appropriate  to  that  problem.4  The  conceptual  views,  in  particular,  are  critical  in 
defining  the  relationships  among  the  military  systems,  warfighters,  and  the  natural  environment. 
The  Systems  Engineering  Concept  Model  (SECM)  methodology  is  being  used  in  several  Navy 
programs  in  addition  to  Ship  Self  Defense  to  capture  and  analyze  all  relevant  entities,  their 
attributes  and  their  interactions  in  a  defined  military  scenario,  first  from  the  operational  or  real 
world  view  and  then  from  the  simulation  perspective. 

REAL  WORLD  VIEW 

The  suite  of  military  systems  involved  in  Ship  Self  Defense  includes  sensors;  weapons  to 
destroy  the  threat  (a  “hard”  kill)  and  weapons  to  decoy  a  threat  (a  “soft  kill”);  and  the 
accompanying  processing  and  control  software  and  hardware  (consoles,  displays  and  computers). 
Hard-kill  weapons  can  include  a  variety  of  missiles.  Soft-kill  weapons  are  decoys  denoted  as 
seduction  or  distraction.  The  particular  components  vary  from  ship  to  ship. 

Figure  1  shows  a  ship  in  a  combat  situation  under  attack  from  an  ASCM  threat.  The  role 
of  the  natural  environment  for  Ship  Self  Defense  is  highlighted  by  this  diagram.  An  ASCM 
operates  in  and  has  its  system  performance  affected  by  the  natural  environment.  The  natural 
environment  also  impacts  the  performance  of  the  ship  and  the  ship’s  sensing  and  defensive 
systems.  First,  the  natural  environment  modifies  the  radio  frequency  (RF)  signals  that  pass 
through  it.  Second,  the  ship  and  the  EW  System  will  pitch  and  roll  in  response  to  wind  and 
waves,  thereby  affecting  the  performance  of  the  Ship  Self  Defense.  The  interactions  between  the 
threat  and  the  ship  occur  exclusively  through  the  natural  environment  until  one  of  three  things 
occur:  1)  the  threat  damages  the  ship,  2)  the  ship’s  weapons  kill  the  threat  (via  a  hard  or  soft 
kill),  or  3)  the  threat  misses  entirely.  All  of  the  actions  noted  above  occur  as  responses  to  the 
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threat  misses  entirely.  All  of  the  actions  noted  above  occur  as  responses  to  the  state  of  the 
common  natural  environment. 


Real  World 
Information  Flow 
Partial  View 


Figure  1.  A  generalized  representation  of  the  relationships  between  the  environment  and 
threat  and  ship  defensive  systems  performances. 


The  conceptual  view  shown  in  Figure  1  is  expanded  when  system  operators  are  included 
as  shown  in  Figure  2.  The  system  operators  receive  and  respond  to  information  provided  by  the 
ship’s  systems,  including  weapons  and  sensors.  The  impact  that  the  external  natural  environment 
has  on  this  information  depends  on  the  particular  ship’s  system  providing  the  information.  Note 
however,  that  the  performance  of  the  system  operators  included  in  Figure  2  are  not  affected 
directly  by  the  external  (i.e.,  outside  the  ship)  natural  environment,  but  their  performance  and 
actions  are  impacted  by  their  perceptions  of  the  natural  environment  as  presented  to  them  by 
their  sensors  and  other  ship  information  systems.  In  addition,  the  system  operators  must  perform 
their  jobs  in  a  microclimate  of  environmental  conditions  inside  the  ship  that  might  have  a 
detrimental  impact  on  their  physical  and  behavioral  performance  (i.e.,  decision  making). 
Capturing  the  details  of  how  the  natural  environment  affects  the  systems  operators  requires  an 
understanding  of  how  human  performance  (decision  making)  functions  in  general. 

CONCEPTUAL  REPRESENTATION  OF  THE  DECISION  MAKING  PROCESS 

In  Chapter  8  of  their  book,  Wickens  and  Hollands  (2000)  have  described  the  decision 
making  process  in  detail,  including  a  diagram  that  shows  the  operations  and  flow  involved.  Blake 
and  Meyer  (2003)  adapted  this  diagram,  which  is  shown  in  Figure  3,  to  demonstrate  where  the 
natural  environmental  processes  and  data  can  appear  in  the  decision-making  process.  In  Figure  3, 
the  components  that  involve  the  natural  environment  are  shown  as  green  ovals.  The  orange  boxes 
show  the  decision  making  processes.  Note  that  Options  Risks  may  also  be  part  of  the  Long  Term 
Memory.  Different  models  and  theories  exist  for  each  stage  shown  -  sensory  processing, 
perception,  long  term  memory,  and  cognition  -  and  for  the  interactions  among  the  stages. 

This  figure  merely  suggests  the  stages  involved  in  decision  making  in  dynamic  scenarios. 
The  decision  making  process  outlined  in  Figure  3  will  be  discussed  in  more  detail  later,  but 
briefly  the  decision  maker  receives  cues  through  the  five  senses,  analyses  those  cues  together 
with  available  memories,  selects  a  response,  and  then  executes  that  response.  The  results  of  the 
response  executed  become  part  of  recent  past  and  the  process  is  repeated. 
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Ship  and  its  Systems 
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Figure  2.  A  generalized  representation  of  the  relationships  between  the  environment,  threat  and 
ship  defensive  systems  performances,  and  human  decision  makers. 

The  decision-making  process  shown  in  Figure  3  may  be  understood  by  comparing  to  the 
United  States  Marine  Corp  statement  of  Observe,  Orient,  Decide,  and  Act  (OODA).  The  left 
hand  column  is  Observe,  the  second  column  from  the  left  is  Orient,  the  second  from  the  right  is 
Decide  and  the  right  hand  column  is  Act.  For  the  Marine  on  the  ground,  the  Cues  (the  light  green 
oval  in  Figure  3)  are  provided  directly  by  the  in-situ  environment.  For  the  System  Operators  in 
Figure  2,  the  Cues  are  provided  by  the  displays  (visual  and  auditory)  from  information  obtained 
by  the  ship  systems  and  affected  by  the  in-situ  environment. 

MILITARY  DECISION  MAKING 

The  decision  making  process  shown  in  Figure  3  is  general  for  human  performance  in  any 
situation  when  environmental  processes  are  relevant.  The  decision  making  process  for  a  single 
System  Operator  is  only  one  part  of  the  overall  Ship  Self  Defense  process  during  a  combat 
situation.  A  representation  of  the  overall  Ship  Self  Defense  processes  in  which  the  roles  of 
human  operators,  automated  response  systems,  and  natural  environmental  processes  is  shown  in 
Figure  4.  The  Blue  symbols  on  the  left  signify  the  data  generated  and  used  by  the  ship  systems 
and  operators,  while  the  Red  symbols  on  the  right  represent  those  of  the  Threat  (no  operators). 
The  decision  making  processes  are  shown  in  the  lower  left  hand  loop  starting  just  with  Blue’s 
Perception  of  the  Environment. 
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Engineering  Psychology  and  Human  Performance 

Christopher  D.  Wickens  and  Justin  G.  Hollands 
Prentice  Hall,  1999.  Adapted  from  Figure  8.1. 
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Figure  3.  A  schematic  representation  of  the  human  decision  making  process  in  which 
environmental  factors  are  considered. 

At  the  start  of  the  engagement,  both  the  blue  systems  and  the  threat  systems  are  impacted 
by  the  state  of  the  environment  as  noted  by  the  symbol  “common  physics  environment”  (natural 
environment  in  Figures  1  and  2).  The  physical  state  of  the  environment  directly  affects  the  threat 
aerodynamically  and  indirectly  via  the  perceived  environment  as  viewed  by  its  sensors.  The 
Threat’s  responses  to  its  perceived  environment  are  driven  by  preprogrammed  tactics  that  are 
embedded  in  the  threat’s  process  and  control  systems.  The  blue  systems  are  impacted  in  a  similar 
fashion.  The  blue  sensors  detect  a  threat  signature  and  generate  the  blue  sensed  threat 
environment  (cues  in  Figure  3)  which  can  be  responded  to  by  either  human  driven  or  automatic 
defense  responses. 

Blue  sensors  not  only  provide  information  about  the  threat  but  also  about  the  in  situ 
natural  environment  as  determined  by  the  onboard  weather  sensors.  All  of  the  observations  of  the 
threat  and  the  natural  environment  form  part  of  the  perceived  environment  that  is  the  basis  of  the 
information  set  used  in  decision  making.  For  combat  situations  in  general,  red  and  blue  forces 
operate  in  a  common  natural  environment,  but  the  two  forces  may  receive  different  information, 
observations  and  forecasts,  about  the  natural  environment  and,  hence,  have  different  perceptions 
of  the  natural  environments  they  operate  in.  These  differences  in  their  perceived  environments 
play  a  critical  role  in  determining  the  progress  and  advantages  in  combat  situations.5 

Blue’s  memory  of  past  engagements,  intel  on  threat  capabilities  and  ship  tactics,  doctrine, 
and  rules  of  engagement  are  either  in  the  human  memory  or  stored  in  readily  accessible 
databases  and  are  considered  along  with  the  perception  of  the  environment  in  an  the  analysis  of 
the  threat  environment.  This  analysis  includes  a  determination  of  what  the  operational  situation  is 
and  what  the  risks  and  options  are  for  employing  the  Ship  Self  Defense  systems.  If  the  weapons 
deployed  eliminate  the  threat,  then  the  engagement  is  over.  If  not,  or  if  additional  threats  appear, 
then  the  systems  are  reengaged. 
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system  which  has  an  EW  operator  in  the  decision  making  loop  but  does  not  have  a  fully 
automated  mode. 

As  the  military  increasingly  uses  M&S  for  training,  analysis  and  acquisition,  the  demand 
has  grown  for  more  robust  models  both  of  military  systems  and  of  the  role  warfighters  play  in 
using  them.  In  recognition  of  this  growing  demand,  the  Defense  Modeling  and  Simulation  Office 
(DMSO)  requested  that  the  National  Academy  of  Sciences  undertake  a  comprehensive  review  of 
the  human  behavior  representation  (HBR)  for  military  simulations.  The  resulting  report  (Pew  and 
Mavor  1998)  describes  a  wide  range  of  HBR  models  both  for  individual  warfighters  and 
organized  units  of  varying  size.  These  HBR  models  cover  various  aspects  of  human  performance 
including  human  decision  making,  situation  awareness,  planning  and  command  and  control. 
HBR  models  range  in  complexity  from  time  delay  models  to  adaptive  learning  models. 
Variability  in  human  response  times  due  to  stress,  workload  and  other  factors  may  be 
incorporated  by  randomly  varying  either  the  response  times  or  the  course  of  action  selected.  In 
the  following  section,  we  will  discuss  how  the  results  of  this  study  can  be  incorporated  in  an 
existing  simulation  system  developed  to  support  the  development  of  Ship  Self  Defense  systems. 

PRA  ASSESSMENT  SIMULATION  TESTBED 

In  the  PRA  Assessment  PS&A,  the  Ship  Self  Defense  systems  are  analyzed  to  identify 
the  functional  capabilities  that  impact  the  PRA  MOE  which  are  then  captured  in  appropriate 
models  of  the  various  systems.  These  models,  in  turn,  are  clustered  into  several  interactive 
simulations,  which  are  linked  over  a  High  Level  Architecture  (HLA)  Run  Time  Infrastructure 
(RTI),  to  form  a  federation  called  the  PRA  Assessment  Simulation  Testbed.  Each  ship  class  will 
have  a  different  testbed,  corresponding  to  the  different  suite  of  military  systems  included.  A 
general  version  of  the  Ship  Self  Defense  Testbed,  shown  in  Figure  5,  models  threats,  a  variety  of 
LPD17  sensors,  ship  tactics,  and  the  natural  environment.  The  display  is  notional  and  does  not 
represent  any  specific  testbed. 

The  ship  sensor  systems  displayed  include  two  RF  active  radars,  the  SPQ-9B  and  the 
SPS-48E,  and  the  EW  System,  SLQ-32A.  The  two  Processing  and  Control  systems,  CEP  and 
SSDS,  are  in  the  Sensor  Fusion  and  Control  System  Federates,  respectively,  shown  in  the  lower 
left  comer.  The  Threat/EA  Federate  in  the  upper  left  comer  of  the  figure  is  the  most  complex  and 
includes  a  threat  model,  a  decoy  model  and  the  relevant  ship  functionalities.  The  decoy  model  in 
the  federate  is  that  of  a  Nulka.  Only  the  key  ship  functionalities  are  included,  the  ship’s  RF 
signature  (radar  cross  section)  and  the  ship  motion.  The  ship  motion,  primarily  orientation  and 
pitch,  affect  the  performances  of  the  ship  self  defense  combat  systems  as  well  as  the  performance 
of  the  approaching  threat.  The  ship’s  hard  kill  weapon  is  the  Rolling  Airframe  Missile  (RAM). 

The  common  physics  environment  data  is  provided  by  the  Scenario  &  Environment 
Federate.  This  federate  also  provides  the  scenario  data  such  as:  initial  ship  position,  date,  time  of 
day,  playbox,  etc.  The  common  physics  data  provided  corresponds  to  the  natural  environment  in 
Figures  1  and  2  and  the  common  physics  environment  in  Figure  4.  This  natural  environment  data 
must  be  physically  and  dynamically  consistent  to  form  a  Verified  and  Validated  Natural 
Environment  Representation  (Hummel  and  Blake  2001).  For  example,  the  wind  direction  and 
speed  must  be  consistent  with  the  wave  height  and  direction.  Further,  this  common  physics 
environment  data  is  provided  to  each  federate  to  ensure  that  each  federate  is  on  “a  level  playing 
field.”  The  perceived  environment,  described  in  the  last  section,  is  that  obtained  by  the  various 
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sensors,  ship  and  threat.  The  models  for  these  sensors  are  contained  within  the  various  federates 
and,  therefore,  the  perceived  environment,  is  developed  within  these  federates  at  runtime. 6 
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Figure  5.  The  Modeling  and  Simulation  Components  that  make  up  the  Ship  Self  Defense 
PRA  Assessment  Simulation  Testbed. 


Many  of  the  models  contained  in  the  various  federates  shown  in  Figure  5  are  legacy 
models  or  are  rehosted  software  for  the  operational  military  systems.  Whenever  appropriate,  the 
models  represent  the  operational  (actual)  military  systems,  running  in  the  fully  automated  mode. 
The  exception  is  the  SLQ-32A,  which  must  have  an  EW  Operator  for  combat  situations.7  Thus, 
the  Testbed  must  contain  an  EW  Operator  model  that  has  the  appropriate  functional  capabilities. 
As  this  EW  Operator  is  developed,  its  requirements  for  the  environment  data  must  be  identified 
so  that  data  can  be  provided  by  the  Scenario  &  Environment  Federate.  At  this  stage  of 
development,  the  SLQ-32A  federate  is  planned  as  a  combination  of  software  models  to  represent 
the  functionalities  in  the  SLQ-32A,  together  with  the  Electronic  Support  Enhancement 
Processing  Unit  as  Hardware-in-the-Loop.  The  EW  Operator  will  be  included  as  one  of  the 
model  components  in  the  SLQ-32A  federate. 

In  Chapter  12  of  the  report  by  Pew  and  Mavor  (1998),  they  recommend  a  methodology 
for  developing  human  behavior  models  and  identify  the  need  to  understand  what  tasks  are  being 
performed  as  the  critical  first  step  in  selecting  or  constructing  the  appropriate  HBR  model.  When 
restated  in  M&S  terminology,  the  critical  first  step  is  to  identify  the  simulation  requirements  for 
the  HBR  model.8 

The  requirements  for  the  Ship  Self  Defense  PRA  Assessment  Simulation  Testbed  are 
lengthy  and  specific  to  each  ship  class.  However,  the  minimum  EW  Operator  model 
requirements  can  be  generalized  as  follows: 
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1 .  The  EW  Operator  model  must  select  the  course  of  action  from  the  cues  provided  by 
the  Testbed  during  runtime. 

2.  The  EW  Operator  model  must  execute  the  course  of  action. 

3.  The  EW  Operator  model  actions  must  occur  at  the  appropriate  time  during  the 
runtime  of  the  Testbed. 

4.  The  EW  Operator  model  actions  must  take  the  appropriate  amount  of  time  to  execute 
during  the  runtime  of  the  Testbed. 

Based  on  these  requirements,  the  EW  Operator  model  should  receive  the  perceived 
environment  as  input  and  will  provide  the  appropriate  course  of  action,  including  timing,  as 
output.  The  emphasis  of  these  requirements  appears  to  be  on  response  selection  and  execution.  In 
actual  combat  situations,  the  EW  Operator  executes  the  course  of  action  through  a  series  of 
keystrokes.  Therefore,  an  appropriate  EW  Operator  model  might  appear  to  be  one  that  selects  the 
course  of  action  depending  on  the  input  and  transmits  that  course  of  action  after  a  time  delay  that 
corresponds  to  the  time  it  takes  an  EW  Operator  to  execute  the  appropriate  series  of  keystrokes. 
As  shown  in  Figure  6,  this  type  of  EW  Operator  model  only  captures  the  time  delay  associated 
with  the  last  step  in  the  decision  making  process,  not  with  the  preceding  steps.  Can  the  time 
delays  associated  with  the  preceding  steps  be  identified  and  captured  in  a  simple  EW  Operator 
model? 


Figure  6.  EW  Operator  model  as  part  of  human  performance  (decision  making). 

In  order  to  identify  the  time  delays  associated  with  each  step  in  the  decision  making 
process  as  well  as  the  link  to  perceived  environment,  each  step  in  Figure  6  must  be  examined  in 
more  detail,  starting  with  sensory  processing  as  shown  in  Figure  7.  The  cues  are  external  to  the 
EW  Operator  and  include  screen  and  casualty  radar  screens,  although  the  combat  situation  is 
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more  complex.  Both  informative  and  extraneous  cues  are  received.  However,  only  some  of  the 
cues  are  passed  on  to  the  perception  stage.  Both  selective  attention  and  clue  filtering  are  used  to 
determine  which  cues  are  perceived.  For  example,  an  operator  who  is  focused  on  a  visual  cue 
(selective  attention)  may  not  be  aware  of  (receive  or  internalize)  an  auditory  cue  that  is  present. 
Or  an  operator  who  is  focused  on  visual  cues  may  be  aware  of  only  a  change  in  the  pattern,  not 
the  background  pattern  itself  (clue  filtering).  Although  not  detailed  in  Figure  7,  clue  filtering  is 
affected  by  expectations  of  what  the  cues  will  be.  These  expectations  are  based,  in  part,  on  the 
scenario  environment,  present  and  past.  Remember  that  for  the  EW  Operator,  the  environment 
refers  to  the  perceived  environment,  not  the  natural  or  common  physics  environment  shown  if 
Figures  3  and  4.  Knowledge  from  previous  experience  (learning)  and  automatic  constraints  on 
attention  affect  selective  attention  and  clue  filtering. 
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Figure  7.  Representation  of  the  processes  in  decision  making  by  EW  Operator  -  the  sensory  processing  step. 


Once  the  cues  have  been  selected  and  sent  to  the  brain,  they  must  be  interpreted  for 
meaning.  For  example,  the  visual  cues  from  the  SLQ-32A  screen  must  be  interpreted  as  to 
whether  or  not  an  emitter  is  present  while  those  from  the  causality  radar  screens  are  interpreted 
as  to  whether  or  not  there  is  a  hard  track  present.  The  processes  that  make  up  this  step  are 
represented  in  Figure  8.  However,  the  hard  track  is  not  merely  a  matter  of  yes  or  no  but  actually 
has  levels  certainty  ranging  from  0-7  depending  on  the  situation.  An  increasing  number  of 
options  for  interpretation  generally  means  an  increasing  time  for  the  perception  stage. 

The  Analysis  stage  is  the  one  most  commonly  associated  with  decision-making.  The 
environment  past  is  part  of  the  information,  along  with  that  perceived  environment  present 
provided  by  the  cues,  that  is  analyzed  to  determine  what  the  combat  situation  is.  The  blue  box  to 
the  right  shows  four  possible  states,  even  without  including  the  rankings  of  0  -  7  for  whether 
there  is  a  hard  track.  Thus,  the  analysis  is  not  automatic  but  requires  a  weighting  of  the 
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information  available.  Analysis  will  take  time,  and  the  amount  of  time  will  vary  with  the 
perceived  combat  situation. 


The  remaining  step  in  decision  making  that  remains  to  be  examined  is  that  for  response 
selection  as  shown  in  Figure  9.  Here,  the  risks  and  options  in  Figure  3  have  been  separated  for 
detailed  discussion.  Remember  that  the  risks  and  options  are  stored  in  the  long  term  memory. 
The  options  are  for  no  launch  of  weapons  or  to  launch  one  or  more  weapons  in  various 
combinations.  Note  that  the  EW  Operator  does  not  directly  control  the  launching  of  all  weapons 
shown.  There  are  risks  for  launching  or  not  launching  weapons  when  a  target  is  detected.  The 
examples  shown  denote  well  documented  combat  situations.  The  USS  Stark  did  not  fire  on  an 
approaching  target,  a  response  selection  that  resulted  in  military  lives  lost.  The  USS  Vincennes 
did  fire  on  a  target  that  turned  out  to  be  a  civilian  transport,  not  a  threat,  with  tragic 
consequences.  Again,  the  availability  of  several  options  along  with  the  knowledge  or  the  risks 
involved  for  any  option  will  increase  the  time  needed  for  this  step. 

The  response  execution  will  modify  the  environment  present,  both  the  common  physics 
environment  and  the  perceived  environment.  The  modifications  to  the  environment  present  can 
be  determined  by  analyzing  the  results  of  the  actions  taken. 
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Figure  9.  The  processes  in  decision  making  by  an  EW  Operator  -  the  response  selection  and  execution  steps. 

FUTURE  EFFORTS  TO  INCORPORATE  HBR  INTO  THE  PRA  TESTBED 

In  the  previous  discussions  we  have  presented  the  methods  by  which  human  behavior  can 
impact  the  decision  making  process  in  ship  self  defense  systems.  Now,  we  want  to  address  the 
issues  associated  with  how  to  implement  these  processes  in  an  actual  simulation  environment,  in 
this  case  the  PRA  Testbed. 

The  detailed  analysis  delineated  in  Figures  7-9  indicate  where  additional  time  delays 
can  (and  will)  occur  in  the  decision  making  process  of  an  EW  Operator.  Further,  the  links 
between  the  environment,  present  and  past,  have  been  highlighted.  The  next  step  is  to  obtain 
actual  data  for  the  time  delays  that  occur  at  each  step  what  other  factors  are  occurring  at  those 
steps  because  they  may  influence  why  the  delays  occurred.  This  can  be  done  with  observations 
combined  with  analytic  studies  (Pew  and  Mavor,  1998,  p.  325).  Two  main  sources  are  potential 
opportunities  for  observing  the  EW  Operator  in  action:  live  exercises  and  simulators. 

Live  exercises  frequently  focus  on  the  performance  of  the  individual  military  systems 
such  as  the  SLQ-32A,  not  on  the  system  operators  or  the  environmental  conditions.  In  fact, 
system  operators  are  not  always  involved  in  the  live  testing.  However,  the  LPD17  PRA 
Assessment  teams  are  working  with  the  teams  involved  in  the  live  testing  to  ensure  that  the 
additional  information  is  recorded.9  These  observations  of  EW  Operators  during  live  testing,  if 
available,  must  be  structured  to  capture  as  much  external  data  (the  natural  environment  and  the 
visual  cues)  and  internal  data  (what  the  operators  are  thinking  and  doing  as  they  view  the 
screens.  Even  under  optimum  observing  conditions,  the  range  of  scenarios,  including  the  natural 
environment,  will  be  limited.  Further,  the  operators  will  be  performing  under  test  conditions,  not 

combat^ondjtjtons.^ave  been  usecj  ;n  0ther  applications  to  observe  human  performance  in  order 
to  develop  cognitive  models.  For  example,  Remington  et  al.  (2004)  describe  observations  of  air 
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traffic  controllers  being  trained  on  simulators  as  they  handle  complex  multi-tasking  situations. 
Simulators  provide  a  wide  range  of  scenarios  and  environmental  conditions  without  the 
possibility  of  serious  consequences  found  in  actual  airport  operations.  However,  the  knowledge 
that  there  will  be  no  serious  consequences  removes  the  element  of  stress  that  affects  the 
performance  of  air  traffic  controllers.  Wickens  and  Hollands  (2000,  Chap.  12)  discuss  in  detail 
the  links  between  stress  and  human  error,  starting  with  the  effect  that  stress  has  on  sensory 
processing.  With  this  limitation  in  mind,  the  simulators  still  offer  one  of  the  few  sources  for 
observing  human  operators  making  decisions  under  a  wide  range  of  conditions. 

Simulators  are  used  to  train  EW  Operators  for  a  wide  range  of  scenario  environment 
condtions.11  The  observations  of  the  EW  Operators  being  trained  must  capture  the  link  between 
their  performance  and  what  they  see  on  their  screens.  Observational  data  of  EW  operators  using 
simulators  can  be  obtained  by  human  observers  or  captured  through  software  applications 
(Femlund  and  Gonzalez  2004). 

Regardless  of  the  source  of  observations  of  EW  operators,  live  testing  or  simulators,  all 
three  aspects  of  the  operations  must  be  captured:  the  human  performance,  the  perceived 
environment  (the  cues  or  screen  displays)  and  the  scenario  natural  environment.  The 
observations  will  not  be  comprehensive  as  it  is  not  possible  to  determine  exactly  how  much  time 
is  spent  on  each  step  in  the  decision  making  process.  However,  if  the  observations  are  made 
carefully,  the  resulting  analysis  can  provide  a  better  determination  of  the  time  required  for  each 
step  in  the  decision  making  process  as  well  as  the  time  to  execute  the  keystrokes.  It  is  expected 
that  the  time  for  the  decision  making  process  as  well  as  the  exact  sequence  of  keystrokes  will 
depend  on  the  perceived  environment  and,  in  turn,  on  the  natural  environment.  This  information 
can  be  used  to  improve  the  EW  operator  keystroke  model  to  more  closely  resemble  what 
happens  in  operational  situations  and  to  provide  consistency  between  human  performance 
(decision  making)  and  the  natural  environment  across  integrated  naval  simulations,  specifically 
the  PRA  Assessment  Testbed. 

The  EW  operator  time  delay  data,  coupled  with  the  environmental  factors  discussed 
previously,  can  be  used  with  a  tool  called  the  Framework  for  Addressing  Cooperative  Extended 
Transactions  (FACET)  to  model  the  complete  end-to-end  Ship  Self  Defense  decision  making 
process.  FACET  was  developed  at  the  Argonne  National  Laboratory  as  part  of  the  Dynamic 
Information  Architecture  System  simulation  framework  system  (Hummel  and  Christiansen 
2000).  FACET  was  developed  to  describe  the  processes  involved  in  course  of  action  develop¬ 
ment  and  execution.  Originally  developed  to  support  the  modeling  of  a  complete  healthcare 
provider,  FACET  was  used  to  capture  the  processes  and  data  used  by  humans  in  all  aspects  in  a 
healthcare  system  including  the  patients  who  are  responding  to  physiological  changes  in  their 
bodies  and  deciding  if  they  need  to  see  a  doctor  to  the  actual  healthcare  providers  who  must 
respond  to  symptoms  described  by  the  patients  and  the  data  that  collect  about  the  patient’s  state 
and  develop  a  course  of  treatment.  FACET  was  developed  from  a  generalized  perspective  and 
can  be  used  to  model  the  social  interactions  between  any  animal  life  form.  FACET  can  also  be 
used  to  model  the  decision  making  processes  of  physics  driven  entities  as  well. 

SUMMARY 

The  US  Navy  is  increasingly  using  M&S  to  supplement  live  exercises  and  testing  for 
training,  analysis  and  acquisition.  The  performance  of  military  systems  in  combat  situations  is 
affected  by  the  performance  of  the  warfighters;  both  in  turn  are  affected  by  the  natural 
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environment.  The  interoperable  simulations  developed  must  reflect  the  functional  relationships 
in  a  consistent  manner  among  the  military  systems,  warfighters  and  natural  environment. 
Specifically,  human  performance  (decision  making)  and  the  scenario  environment  are  linked  in 
the  real  world  and  therefore  must  be  consistent  across  integrated  naval  simulations.  In  this  paper 
we  have  presented  the  first  steps  in  defining  the  requirements  for  human  performance 
representation  and  natural  environment  representation  that  is  consistent  across  integrated  naval 
simulations. 

The  Ship  Self  Defense  PRA  Assessment  PS&A  includes  the  methodology,  SECM,  for 
establishing  consistency  between  the  military  system  models  and  the  natural  environment 
representation  data.  This  methodology  has  been  extended  to  demonstrate  how  consistency  can  be 
established  for  human  performance  (decision  making)  as  well.  At  the  conceptual  level  the  steps 
followed  are: 

•  Examine  the  relationships  for  Ship  Self  Defense  in  combat  situations. 

•  Examine  the  general  human  performance  (decision  making)  process  with  the  effect  of 
the  environment  included. 

•  Develop  the  military  decision  making  process  for  Ship  Self  Defense  in  a  combat 
situation. 

•  Analyze  the  PRA  Assessment  Testbed  to  determine  the  requirements  for  an  EW 
operator  model.  Based  on  the  requirements,  a  keystroke  model  will  be  used. 
Keystroke  models  generally  capture  only  the  response  execution  actions,  not  the 
complete  decision  making  process,  including  the  impact  of  the  natural  environment 

•  5®iftfil)1tfifee<!lcision  making  process  specifically  for  a  keystroke  EW  operator  model 
to  determine  the  links  among  the  steps  in  decision  making,  the  cues  provided  by  the 
military  system  sensors,  and  the  common  physics  environment. 

The  analysis  indicates  that  the  keystroke  model  for  an  EW  operator  should  incorporate 
time  delays  and  keystroke  sequences  that  depend  directly  on  the  cues  (perceived  environment) 
that,  in  turn,  depend  on  the  natural  environment  representation  provided.  Even  a  simple  time 
delay  model  of  human  performance  can  be  linked  to  the  scenario  environment. 

The  next  steps,  reserved  for  future  work  will  be  to  obtain  the  data  needed  to  establish  the 
values  for  the  time  delays  and  the  keystroke  sequences  for  various  scenario  environment 
conditions.  Such  data  can  be  obtained  from  live  testing  and  from  simulators  used  to  train  EW 
operators.  Observations  of  EW  operators  must  be  structured,  if  possible,  to  identify  the  mental 
processes  in  decision  making  as  affected  by  the  scenario  environment  presented. 


ENDNOTES 

1  The  authors  welcome  additional  information  about  such  studies. 


2  “AAW  Capstone  Requirements  Document”  Chief  of  Naval  Operations,  5  February  1996. 


3  “Navy  Ship  Self  Defense  Combat  Systems  Engineering  Probability  of  Raid  Annihilation  (PRA) 
Assessment  Process  Standards  and  Architecture  (PS&A),  Version  1.1,  May  2003  may  be 
obtained  from  VisiTech,  Ltd,  535  E.  Braddock  Road,  Alexandria  VA,  22314,  Email: 
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4  The  Program  Executive  Office  (PEO)  Integrated  Warfare  Systems  (IWS)  Systems  Engineering 
Process,  which  does  address  Modeling  &  Simulation,  is  the  guide  for  the  PRA  Assessment 
PS&A. 

5  The  difference  between  the  “common  physics  environment”  in  which  the  military  systems 
operate  and  the  “perceived”  environment  which  the  systems  operators  use  in  decision  making  as 
well  as  the  roles  that  both  types  of  information  can  play  in  M&S  has  not  received  sufficient 
recognition.  See  Lucas  et  al  (2000)  for  additional  information. 

6  Many  of  the  high  fidelity  physics  models  are  run  ahead  of  time  and  the  results  are  stored  in 
databases  that  are  accessed  at  runtime. 

n 

Information  provided  by  Kevin  Brown  (US  Naval  Research  Laboratory  -  DC),  2003, 
unpublished  paper. 

8  The  lead  author  thanks  Dr.  Ruth  Willis  (Advanced  Information  Technology  Branch,  US  Naval 
Research  Laboratory  -  DC),  2002,  for  discussions  on  human  behavior  representation  models.  Dr. 
Willis  emphasized  both  the  importance  of  and  difficulties  in  establishing  requirements  for  HBR 
models  as  a  first  step  in  developing  appropriate  HBR  models. 

9  The  lead  author  thanks  Duane  Coleman  (Naval  Surface  Warfare  Center  -  Corona  Division)  for 
this  information. 

10  The  authors  thank  Bruce  E.  Eckstein  (FAA,  Washington,  DC)  for  discussions  about  using 
simulators  in  training  air  traffic  controllers. 

11  One  example  is  the  EWPro™.  http://www.rdsi.com.  Last  accessed  on  1 1  November  2004. 
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DESCRIPTORS 

Human  performance 

Decision  making 

Natural  environment 

Perceived  environment 

Probability  of  Raid  Annihilation  (PRA) 

Modeling  and  Simulation  (M&S) 

Ship  Self  Defense  (SSD) 

LIST  OF  ACRONYMS 


AAW 

Anti  Air  Warfare 

ASCM 

Anti  Ship  Cruise  Missile 

BRIMS 

Behavior  Representation  in  Modeling  and  Simulation 

CEP 

Cooperative  Engagement  Processor 

EW 

Electronic  Warfare 

FACET 

Framework  for  Addressing  Cooperative  Extended  Transactions 

HBR 

Human  Behavior  Response 

HLA 

High  Level  Architecture 

LPD 

Amphibious  Transport  Dock 

M&S 

Modeling  and  Simulation 

MOE 

Measures  of  Effectiveness 

MORSS 

Military  Operations  Research  Society  Symposium 

OODA 

Observe,  Orient,  Decide,  and  Act 

OPEVAL 

Operations/Operational  Evaluation 

PRA 

Probability  of  Raid  Annihilation 

PS&A 

Process  Standards  and  Architecture 

RAM 

Rolling  Airframe  Missile 

RF 

Radio  Frequency 

RTI 

Run  Time  Infrastructure 

SECM 

Systems  Engineering  Concept  Model 

SSD 

Ship  Self  Defense 

SSDS 

Ship  Self-Defense  System 
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